It is premature to regard the ego-depletion effect as “Too Incredible”

The “strength” model conceptualizes selfcontrol as a limited resource (Baumeister et al., 1998). Individuals are able to exert self-control, but only for a limited period after which capacity declines leading to reduced self-control capacity; a state known as ego-depletion. The model has generated a sizable literature confirming the ego-depletion effect in multiple spheres. Our meta-analysis of published ego-depletion studies computed a medium-sized effect (d = 0.62) across 198 tests (Hagger et al., 2010). Carter andMcCullough (2013) recently applied analyses aimed at testing for publication bias to our data including Schimmack’s (2012) incredibility index and two regression techniques (Egger et al., 1997; Moreno et al., 2009). Regression analyses indicated that the ego-depletion effect was substantially smaller than reported in our analysis and may even be zero, and the incredibility index indicated low statistical power and the chances of finding so many significant effects improbable. They concluded that the ego-depletion effect is subject to considerable publication bias and questioned whether it is a “real” effect at all. We replicated these analyses and found similar results. We have made our analyses available to download from the open-access Dryad Digital Repository (Hagger and Chatzisarantis, 2014). We thank Carter and McCullough (2013) raising the issue of bias. We take this opportunity to present some alternative conclusions to the ones they presented. We agree that journal editors should be more judicious in demanding bias tests in meta-analyses, but believe that that recommendation does not resolve the problem of interpreting the bias. An important addendum to the regression analyses is that the bias detected by a significant regression line cannot be definitively attributed to publication bias. Sterne et al. (2000, 2001) suggest that such bias could be attributed to a number of possible sources. Instead, they use the term “small study” effect; the tendency for smaller studies to report larger effect sizes. One possible reason would be due to publication bias: journals tending to favor the publication of small studies with statistically significant results and disproportionately large effect sizes. However, the findings may also be due to methodological inadequacies or true heterogeneity in the effect. A definitive response to resolving the nature of bias detected by these methods (i.e., whether it is publication bias or other source of bias that causes a “small study effect”) would be to demand authors conducting meta-analyses be diligent in the pursuit of “fugitive literature”: unpublished studies with null findings, or findings that conflict with the commonlyaccepted paradigm, that Rosenthal (1994) eloquently predicted would reside in the “file drawers” of researchers who could not get them published. In the case of ego-depletion, a unique contribution would be to identify unpublished studies including those with null or negative effects, as well as studies that have since been published, and recalculate the metaanalytic effect size. Such an undertaking would not only yield a more robust effect size ostensibly independent of publication bias but also be informative as to whether the “small study effect” detected in the analyses was due to publication bias, other forms of bias, or true heterogeneity. We encourage researchers to make their replications of ego-depletion studies freely available to aid future meta-analyses. We would also like to express concerns regarding Carter and McCullough’s prediction, based on their regression analyses, that the ego-depletion effect may be zero. This prediction was based on the intercept of the regression of the ego-depletion effect size on precision. However, if the true ego-depletion effect size is zero or close to it, one would expect the effect sizes in the literature to be randomly distributed in both positive and negative directions about zero. If this is the case, then where are those negative findings? There are scant few ego-depletion experiments that have found opposite effects, i.e., an improvement in second-task performance after engaging in an initial self-control task, let alone null effects. Given the intensiveness of research in this field, would it not be reasonable to expect to have seen the negative findings published? The absence of these effects creates a problem for the claim that